Increasingly, more and more tests are being administered using electronic resources. Students may take examinations through online testing systems which may provide adaptive testing.
In one system, U.S. Pat. No. 7,628,614 describes a method for estimating examinee attribute parameters in cognitive diagnosis models. The patent describes estimating for each assessment one or more item parameters and also describes tracking item responses by considering various examinee parameters. Although many systems determine a user's responses to an examination question item, those systems do not consider user behavior that may be related to the user's responses within the examination or more importantly user's proficiency. By focusing on solely examination question item response data, those systems fail to consider the wealth of information that may be collected within a computerized learning system.
Many assessment systems only track the ability of the user and neither consider nor attempt to improve the performance of the students. These systems also fail to recognize the potential uses for the vast amount of information that may be detected while a user accesses a learning system. Moreover, some current methods of interpreting data are unable to cope with the high volume and plethora in types of traffic information that potentially may be collected during an educational or assessment session. Large amounts of data and types of activity traffic may prove to be difficult to effectively model and process in order to discover useful information. Many assessment systems therefore only track data associated with user's responses during examination.
Also, it may be difficult and time consuming for instructors to create new examinations and furthermore prepare students by creating additional practice examinations. Traditional methods involve picking questions from a textbook or from past exam in an attempt to make a mock exams and even the examination themselves. Such methods are not only onerous to instructors but more importantly, especially in a multi-section course setup, can lead to a bias in picking questions similar to an already known final in their instruction or mock exams. Some basic methods have been developed by other to generate random problem sets (i.e. different questions), but such methods are too primitive for actual use in creating a well balance exam in terms of distribution of difficulty and variety of questions contained, thus have only be used in low stakes homework. Furthermore full solutions are almost never provided, only the final solution. The lack of a method for simulation exam questions with full solution is detrimental in helping students prepare and study in an effective and smarter manner.
In any sizeable group of students, typically, students will struggle in different study areas or be at different proficiency levels for the same learning objective. An instructor will typically tailor instruction to the average student group leaving weaker students frustrated and lost and stronger students unchallenged. The traditional education system fails to tailor education items and examination items for the needs of each individual student.
Traditionally, courses are designed by professors with experience on what topics must be covered during a semester and in what proportions. Information about student progress can only be found during examinations and in traditional systems, this information is not easily presented and cannot be used to determine the effectiveness and quality of specific learning resources, courseware, textbooks, activities and schedule.